Skeleton-Based Square Grid for Human Action Recognition With 3D Convolutional Neural Network

Convolutional neural networks (CNNs) can effectively handle grid-structured data but not dynamic skeletons, which are usually expressed as graph structures. In this study, we first propose a skeleton-based square grid (SSG) for transforming skeletons into three-dimensional (3D) so that CNNs be applied to such data. Each SSG contains joint-based (JSG) and rigid-based (RSG) based on intrinsic extrinsic dependencies of various body parts, respectively. Next, enhance the ability deep features capture correlations among 3D data, two-stream CNN is constructed learn spatiotemporal using JSG RSG sequences. Finally, introduce soft attention model selectively focuses informative parts in skeleton We validate our terms action recognition three datasets: NTU RGB+D, Kinetics Motion, SBU Kinect Interaction datasets. Our experimental results demonstrate effectiveness proposed approach well its superior performance when compared with those state-of-the-art methods.